{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.网络结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lenet是一个 7 层的神经网络，包含 2 个卷积层，2 个池化层，2 个全连接层，1个输出层。其中所有卷积层的卷积核都为 5x5，步长为1，池化方法都为平均池化，激活函数为 Sigmoid\n",
    "\n",
    "![image-20240807153844641](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20240807153844641.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.代码实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:1, loss:2.3133173278654056, accuracy:0.1\n",
      "epoch:2, loss:2.1397135971959975, accuracy:0.5336\n",
      "epoch:3, loss:0.841501203490727, accuracy:0.7413\n",
      "epoch:4, loss:0.6085703329109688, accuracy:0.7648\n",
      "epoch:5, loss:0.514386103478576, accuracy:0.799\n",
      "epoch:6, loss:0.46315956585951196, accuracy:0.8252\n",
      "epoch:7, loss:0.43332896392736864, accuracy:0.7932\n",
      "epoch:8, loss:0.4083172631606873, accuracy:0.8443\n",
      "epoch:9, loss:0.38466112096426586, accuracy:0.8504\n",
      "epoch:10, loss:0.3698408588099836, accuracy:0.8428\n",
      "epoch:11, loss:0.35436571816772794, accuracy:0.8488\n",
      "epoch:12, loss:0.341442651014084, accuracy:0.862\n",
      "epoch:13, loss:0.3303176119828275, accuracy:0.8566\n",
      "epoch:14, loss:0.3211779908012988, accuracy:0.8647\n",
      "epoch:15, loss:0.31307388120876956, accuracy:0.8642\n",
      "epoch:16, loss:0.30497926978795514, accuracy:0.8778\n",
      "epoch:17, loss:0.2981931160507934, accuracy:0.8761\n",
      "epoch:18, loss:0.29181672924998475, accuracy:0.8745\n",
      "epoch:19, loss:0.2866834082773754, accuracy:0.8748\n",
      "epoch:20, loss:0.2795482828482382, accuracy:0.8786\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABB8UlEQVR4nO3deZwU9Z3/8XfPfc8Ac8MwgHIMMByi4pADRCLixRiDhBhRIyYa+K0EfahsBEzcXUyMrrtKRE2UZIlnoqBCVETRCHggjHIMCMg5zMkxw9xH1++PspvpuXuu6uP1fDzqQXV3VfenLNp+863v91s2wzAMAQAAWCTA6gIAAIB/I4wAAABLEUYAAIClCCMAAMBShBEAAGApwggAALAUYQQAAFiKMAIAACwVZHUBHWG323XixAlFR0fLZrNZXQ4AAOgAwzB09uxZpaamKiCg9fYPrwgjJ06cUFpamtVlAACATjh27JgGDBjQ6uteEUaio6MlmQcTExNjcTUAAKAjysrKlJaW5vwdb41XhBHHpZmYmBjCCAAAXqa9LhZ0YAUAAJYijAAAAEsRRgAAgKW8os8IAMC3GYah+vp6NTQ0WF0K3BAYGKigoKAuT7tBGAEAWKq2tlb5+fmqrKy0uhR0QkREhFJSUhQSEtLp9yCMAAAsY7fbdejQIQUGBio1NVUhISFMbuklDMNQbW2tiouLdejQIQ0dOrTNic3aQhgBAFimtrZWdrtdaWlpioiIsLocuCk8PFzBwcE6cuSIamtrFRYW1qn3oQMrAMBynf0XNazXHeeOsw8AACxFGAEAAJYijAAA0AlTpkzRwoULrS7DJxBGAACApfw7jGzcKF19tVRdbXUlAAD4Lf8NIxUV0pw50rp10o03Ssz6BwDWMwzz/89WLIbR6bJPnz6tuXPnqk+fPoqIiNCMGTO0f/9+5+tHjhzRNddcoz59+igyMlKjRo3S+vXrnfveeOONSkhIUHh4uIYOHarnn3++y/8pvYn/zjMSGSm98oo0fbr02mvSggXSH/8oMdkOAFinslKKirLms8vLzd+GTrjlllu0f/9+vfHGG4qJidF9992nK6+8Unv27FFwcLDmz5+v2tpaffTRR4qMjNSePXsU9e1xLlmyRHv27NE///lPxcfH68CBA6qqqurOI/N4/htGJGnKFOlvf5NuuEFauVJKTZWWLLG6KgCAF3GEkM2bN2vSpEmSpL/97W9KS0vTmjVrNGvWLB09elTXX3+9MjMzJUlDhgxx7n/06FGNHz9eF154oSRp0KBBvX4MVvPvMCJJP/qR9MQTZsvI0qVSSoo0b57VVQGAf4qIMFsorPrsTsjNzVVQUJAmTpzofK5fv34aPny4cnNzJUn/9m//pjvvvFPvvvuupk2bpuuvv15jxoyRJN155526/vrrtX37dl1++eXKzs52hhp/4b99RhqbP1/69a/N9V/8QnrjDWvrAQB/ZbOZl0qsWHrwMv28efP0zTff6KabbtLOnTt14YUX6oknnpAkzZgxQ0eOHNGvfvUrnThxQpdddpnuueeeHqvFExFGHB56SPrZzyS7XZo9W9qyxeqKAABeICMjQ/X19fr000+dz508eVL79u3TyJEjnc+lpaXpjjvu0Guvvaa7775bzz77rPO1hIQE3XzzzVq9erUef/xxPfPMM716DFYjjDjYbNLTT58b6nv11dKePVZXBQDwcEOHDtXMmTN1++236+OPP9aXX36pn/70p+rfv79mzpwpSVq4cKHeeecdHTp0SNu3b9cHH3ygjIwMSdLSpUu1du1aHThwQLt379Zbb73lfM1fEEYaCwqSXn5ZuuQS6fRp6YorpOPHra4KAODhnn/+eU2YMEFXX321srKyZBiG1q9fr+DgYElSQ0OD5s+fr4yMDF1xxRUaNmyY/vjHP0qSQkJCtHjxYo0ZM0bf//73FRgYqJdeesnKw+l1NsPowsDqXlJWVqbY2FiVlpYqJiam5z/w5EnpO9+R9u2TRo2S/vUvqU+fnv9cAPAz1dXVOnTokAYPHtzp28/DWm2dw47+ftMy0pJ+/aR33jGH+u7eLV17reRnY74BAOgthJHWpKdLb78txcZKH38s/eQnzNIKAEAPIIy0JTNTWrtWCg2V1qwxhwB7/lUtAAC8CmGkPZMnm7O0OkbbPPSQ1RUBAOBTCCMdcf310pNPmuvLlkl+Nv4bAICeRBjpqF/+UnrgAXP9zjvNyzcAAKDLCCPu+O1vpdtuM2dp/fGPpc2bra4IAACvRxhxh81m3t33mmvOzdK6e7fVVQEA4NUII+4KCpJeeknKypLOnDFnaT12zOqqAADwWoSRzoiIkN58U8rIMKeLv+IKc/p4AADgNsJIZ/XrZ06Klppq3lCPWVoBABaqq6uzuoROI4x0xcCBrrO0zpkj1ddbXRUAoBe8/fbb+u53v6u4uDj169dPV199tQ4ePOh8/fjx45ozZ4769u2ryMhIXXjhhfr000+dr7/55pu66KKLFBYWpvj4eF133XXO12w2m9asWePyeXFxcVq1apUk6fDhw7LZbHr55Zc1efJkhYWF6W9/+5tOnjypOXPmqH///oqIiFBmZqZefPFFl/ex2+36/e9/r/PPP1+hoaEaOHCg/vM//1OSNHXqVC1YsMBl++LiYoWEhGjjxo3d8Z+tRYSRrsrMlN54w5ylde1aZmkFgC4wDEO1tbWWLO7eN7aiokKLFi3Stm3btHHjRgUEBOi6666T3W5XeXm5Jk+erLy8PL3xxhv68ssvde+998put0uS1q1bp+uuu05XXnmlduzYoY0bN+riiy92+7/X/fffr7vuuku5ubmaPn26qqurNWHCBK1bt067du3Sz3/+c91000367LPPnPssXrxYDz/8sJYsWaI9e/bohRdeUFJSkiRp3rx5euGFF1RTU+PcfvXq1erfv7+mTp3qdn0dxV17u8trr0mzZpnDfh980JwcDQDQpqZ3fK2trdXy5cstqWXx4sUKCQnp9P4lJSVKSEjQzp07tWXLFt1zzz06fPiw+vbt22zbSZMmaciQIVq9enWL72Wz2fT6668rOzvb+VxcXJwef/xx3XLLLTp8+LAGDx6sxx9/XHfddVebdV199dUaMWKE/vCHP+js2bNKSEjQk08+qXnz5jXbtrq6WqmpqVq5cqVuuOEGSdLYsWP1wx/+UMta+V3jrr2e5Ic/lFasMNcffNCcOh4A4LP279+vOXPmaMiQIYqJidGgQYMkSUePHlVOTo7Gjx/fYhCRpJycHF122WVdruHCCy90edzQ0KCHHnpImZmZ6tu3r6KiovTOO+/o6NGjkqTc3FzV1NS0+tlhYWG66aab9Nxzz0mStm/frl27dumWW27pcq1tCerRd/c3d9whnThh3r/ml7+UkpKkRqkWANC24OBgLV682LLPdsc111yj9PR0Pfvss0pNTZXdbtfo0aNVW1ur8PDwNvdt73WbzdbsslFLHVQjIyNdHj/yyCP6n//5Hz3++OPKzMxUZGSkFi5cqNra2g59rmReqhk3bpyOHz+u559/XlOnTlV6enq7+3UFLSPd7Te/kebNMy/XzJljdmwFAHSIzWZTSEiIJYvNZutwnSdPntS+ffv0wAMP6LLLLlNGRoZON5riYcyYMcrJydGpU6da3H/MmDFtdghNSEhQfn6+8/H+/ftVWVnZbl2bN2/WzJkz9dOf/lRjx47VkCFD9PXXXztfHzp0qMLDw9v87MzMTF144YV69tln9cILL+hnP/tZu5/bVYSR7mazSU89ZQ71ra6WfvQjyYuHWwEAmuvTp4/69eunZ555RgcOHND777+vRYsWOV+fM2eOkpOTlZ2drc2bN+ubb77RP/7xD23dulWStGzZMr344otatmyZcnNztXPnTv3ud79z7j916lQ9+eST2rFjh7Zt26Y77rijQy03Q4cO1YYNG7Rlyxbl5ubqF7/4hQoLC52vh4WF6b777tO9996rv/71rzp48KA++eQT/fnPf3Z5n3nz5unhhx+WYRguo3x6CmGkJwQFSS++aI6wKSw0J0YDAPiMgIAAvfTSS/riiy80evRo/epXv9IjjzzifD0kJETvvvuuEhMTdeWVVyozM1MPP/ywAgMDJUlTpkzRq6++qjfeeEPjxo3T1KlTXUa8PProo0pLS9P3vvc9/eQnP9E999yjiIiIdut64IEHdMEFF2j69OmaMmWKMxA1tmTJEt19991aunSpMjIyNHv2bBUVFblsM2fOHAUFBWnOnDnNOqX2BEbT9KRhw6T9+6UPPpCmTLG6GgDwOG2NxIB1Dh8+rPPOO0+ff/65Lrjggja3ZTSNpxs40PzzyBFr6wAAoAPq6upUUFCgBx54QJdcckm7QaS7EEZ6kqP3MWEEAOAFNm/erJSUFH3++edauXJlr30uQ3t7EmEEAOBFpkyZ4vZMtN2BlpGe5Agj3042AwAAmiOM9CRaRgAAaBdhpCc1bhn59uZIAIDmvGBgJ1rRHeeOMNKT+vc3J0GrqZGajOEGAJybgr0js4vCMznOnbvT6TdGB9aeFBIipaZKeXnmpZrkZKsrAgCPEhgYqLi4OOekWxEREW5Nyw7rGIahyspKFRUVKS4uzjmhW2cQRnpaevq5MDJxotXVAIDHSf72H2pNZwGFd4iLi3Oew84ijPS09HRpyxZG1ABAK2w2m1JSUpSYmNjinWnhuYKDg7vUIuJAGOlpjKgBgA4JDAzslh82eB86sPY0wggAAG0ijPQ07k8DAECbCCM9jZYRAADaRBjpaY4wUlpqLgAAwIVbYWT58uW66KKLFB0drcTERGVnZ2vfvn3t7vfqq69qxIgRCgsLU2ZmptavX9/pgr1OVJTUt6+5zogaAACacSuMfPjhh5o/f74++eQTbdiwQXV1dbr88stVUVHR6j5btmzRnDlzdNttt2nHjh3Kzs5Wdna2du3a1eXivQaXagAAaJXN6MKk8sXFxUpMTNSHH36o73//+y1uM3v2bFVUVOitt95yPnfJJZdo3LhxWrlyZYc+p6ysTLGxsSotLVVMTExny7VOdra0dq305JPS/PlWVwMAQK/o6O93l/qMlH7bB6Kv4zJEC7Zu3app06a5PDd9+nRt3bq11X1qampUVlbmsng1WkYAAGhVp8OI3W7XwoUL9Z3vfEejR49udbuCggIlJSW5PJeUlKSCgoJW91m+fLliY2OdS1paWmfL9AyEEQAAWtXpMDJ//nzt2rVLL730UnfWI0lavHixSktLncuxY8e6/TN6lSOM0IEVAIBmOjUd/IIFC/TWW2/po48+0oABA9rcNjk5WYWFhS7PFRYWtnlTndDQUIWGhnamNM9EywgAAK1yq2XEMAwtWLBAr7/+ut5//30NHjy43X2ysrK0ceNGl+c2bNigrKws9yr1Zo4wkp8v1dRYWwsAAB7GrTAyf/58rV69Wi+88IKio6NVUFCggoICVVVVObeZO3euFi9e7Hx811136e2339ajjz6qvXv36sEHH9S2bdu0YMGC7jsKTxcfL4WHm+vefskJAIBu5lYYeeqpp1RaWqopU6YoJSXFubz88svObY4ePar8/Hzn40mTJumFF17QM888o7Fjx+rvf/+71qxZ02anV59js3GPGgAAWuFWn5GOTEmyadOmZs/NmjVLs2bNcuejfE96urRvH2EEAIAmuDdNb2FEDQAALSKM9BZG1AAA0CLCSG8hjAAA0CLCSG+hAysAAC0ijPQWR8vIsWOS3W5tLQAAeBDCSG/p318KDJTq6szJzwAAgCTCSO8JCjIDicSIGgAAGiGM9CY6sQIA0AxhpDcRRgAAaIYw0psYUQMAQDOEkd5EywgAAM0QRnoTU8IDANAMYaQ3NW4Z6cBNBwEA8AeEkd7k6DNy9qx05oylpQAA4CkII70pIkJKSDDX6TcCAIAkwkjvY0QNAAAuCCO9jRE1AAC4IIz0NkbUAADggjDS22gZAQDABWGktxFGAABwQRjpbYQRAABcEEZ6m2M0TVGRVFVlbS0AAHgAwkhv69tXiow01+nECgAAYaTX2WyMqAEAoBHCiBXoNwIAgBNhxAqEEQAAnAgjViCMAADgRBixAvenAQDAiTBiBTqwAgDgRBixgiOMHD8uNTRYWwsAABYjjFghJUUKCpLq66UTJ6yuBgAASxFGrBAYKKWlmev0GwEA+DnCiFUYUQMAgCTCiHUYUQMAgCTCiHUYUQMAgCTCiHW4TAMAgCTCiHUIIwAASCKMWKdxGDEMa2sBAMBChBGrOIb2VlZKJ09aWwsAABYijFglLExKSjLXuVQDAPBjhBErMaIGAADCiKXoxAoAAGHEUoQRAAAII5YijAAAQBixFGEEAADCiKUc96ehAysAwI8RRqzkaBkpKZEqKqytBQAAixBGrBQXJ8XEmOu0jgAA/BRhxGr0GwEA+DnCiNUIIwAAP0cYsRphBADg5wgjVmNEDQDAzxFGrEbLCADAzxFGrEYYAQD4OcKI1RxhJC9PqquzthYAACxAGLFaUpIUEiLZ7WYgAQDAzxBGrBYQcK4TK5dqAAB+iDDiCRhRAwDwY4QRT0AnVgCAHyOMeALCCADAjxFGPAFhBADgxwgjnoAwAgDwY4QRT9C4A6thWFsLAAC9jDDiCdLSJJtNqq6WioutrgYAgF5FGPEEISFSSoq5zqUaAICfcTuMfPTRR7rmmmuUmpoqm82mNWvWtLn9pk2bZLPZmi0FBQWdrdk30W8EAOCn3A4jFRUVGjt2rFasWOHWfvv27VN+fr5zSUxMdPejfRthBADgp4Lc3WHGjBmaMWOG2x+UmJiouLg4t/fzG4QRAICf6rU+I+PGjVNKSop+8IMfaPPmzW1uW1NTo7KyMpfF5zElPADAT/V4GElJSdHKlSv1j3/8Q//4xz+UlpamKVOmaPv27a3us3z5csXGxjqXtLS0ni7TerSMAAD8lM0wOj+xhc1m0+uvv67s7Gy39ps8ebIGDhyo//u//2vx9ZqaGtXU1Dgfl5WVKS0tTaWlpYqJielsuZ5t1y4pM1Pq00c6dcrqagAA6LKysjLFxsa2+/vtdp+R7nDxxRfr448/bvX10NBQhYaG9mJFHsDRMnL6tHT2rBQdbW09AAD0EkvmGcnJyVGKY14NmKKjzVYRiUs1AAC/4nbLSHl5uQ4cOOB8fOjQIeXk5Khv374aOHCgFi9erLy8PP31r3+VJD3++OMaPHiwRo0aperqav3pT3/S+++/r3fffbf7jsJXpKebLSNHjkijR1tdDQAAvcLtMLJt2zZdeumlzseLFi2SJN18881atWqV8vPzdbTRiJDa2lrdfffdysvLU0REhMaMGaP33nvP5T3wrYEDpZwcRtQAAPxKlzqw9paOdoDxev/2b9ITT0j33Sc9/LDV1QAA0CUd/f3m3jSehOG9AAA/RBjxJIQRAIAfIox4EsIIAMAPEUY8iSOM5OdLtbXW1gIAQC8hjHiShAQpLEwyDOn4caurAQCgVxBGPInNdu6GeVyqAQD4CcKIp6HfCADAzxBGPA1hBADgZwgjnoYwAgDwM4QRT+MII0wJDwDwE4QRT0MHVgCAnyGMeJrGLSN2u7W1AADQCwgjnqZ/fykgwJz0rLDQ6moAAOhxhBFPExxsBhKJSzUAAL9AGPFEdGIFAPgRwognYngvAMCPEEY8ESNqAAB+hDDiiWgZAQD4EcKIJyKMAAD8CGHEExFGAAB+hDDiiRx9RsrKpNJSa2sBAKCHEUY8UWSkFB9vrtM6AgDwcYQRT8WIGgCAnyCMeCr6jQAA/ARhxFMRRgAAfoIw4qkIIwAAP0EY8VTcnwYA4CcII56KlhEAgJ8gjHgqx2iaggKputraWgAA6EGEEU/Vr58UEWGuHztmbS0AAPQgwoinstm4VAMA8AuEEU9GJ1YAgB8gjHgyWkYAAH6AMOLJCCMAAD9AGPFk3J8GAOAHCCOejJYRAIAfIIx4MkcYOXZMamiwthYAAHoIYcSTpaZKQUFSfb05+RkAAD6IMOLJAgOlAQPMdS7VAAB8FGHE09FvBADg4wgjno4RNQAAH0cY8XS0jAAAfBxhxNMRRgAAPo4w4um4Pw0AwMcRRjxd45YRw7C2FgAAegBhxNOlpZl/lpdLp09bWwsAAD2AMOLpwsOlxERznX4jAAAfRBjxBnRiBQD4MMKIN6ATKwDAhxFGvAEtIwAAH0YY8QaEEQCADyOMeAPCCADAhxFGvAH3pwEA+DDCiDdwtIwUF0uVldbWAgBANyOMeIO4OCk62lw/dszSUgAA6G6EEW9gs9FvBADgswgj3oIwAgDwUYQRb0EnVgCAjyKMeAtaRgAAPoow4i0IIwAAH0UY8RbcnwYA4KMII97CEUaOH5fq662tBQCAbkQY8RbJyVJwsNTQIJ04YXU1AAB0G8KItwgIkNLSzHX6jQAAfAhhxJvQiRUA4IPcDiMfffSRrrnmGqWmpspms2nNmjXt7rNp0yZdcMEFCg0N1fnnn69Vq1Z1olTQiRUA4IvcDiMVFRUaO3asVqxY0aHtDx06pKuuukqXXnqpcnJytHDhQs2bN0/vvPOO28X6PVpGAAA+KMjdHWbMmKEZM2Z0ePuVK1dq8ODBevTRRyVJGRkZ+vjjj/Xf//3fmj59ursf798IIwAAH9TjfUa2bt2qadOmuTw3ffp0bd26tdV9ampqVFZW5rJAhBEAgE/q8TBSUFCgpKQkl+eSkpJUVlamqqqqFvdZvny5YmNjnUuaYxSJv2t8fxrDsLYWAAC6iUeOplm8eLFKS0udy7Fjx6wuyTM4QllVlVRSYm0tAAB0E7f7jLgrOTlZhYWFLs8VFhYqJiZG4eHhLe4TGhqq0NDQni7N+4SGSikpUn6+OaImIcHqigAA6LIebxnJysrSxo0bXZ7bsGGDsrKyevqjfRP9RgAAPsbtMFJeXq6cnBzl5ORIMofu5uTk6Oi3c18sXrxYc+fOdW5/xx136JtvvtG9996rvXv36o9//KNeeeUV/epXv+qeI/A3hBEAgI9xO4xs27ZN48eP1/jx4yVJixYt0vjx47V06VJJUn5+vjOYSNLgwYO1bt06bdiwQWPHjtWjjz6qP/3pTwzr7SzCCADAx7jdZ2TKlCky2hjJ0dLsqlOmTNGOHTvc/Si0pPGIGgAAfIBHjqZBG2gZAQD4GMKIt+H+NAAAH0MY8TaOMHLypFRRYW0tAAB0A8KIt4mJkeLizHUu1QAAfABhxBvRbwQA4EMII96IETUAAB9CGPFGdGIFAPgQwog34jINAMCHEEa8EWEEAOBDCCPeiDACAPAhbk8HDw/gCCMnTkh1dVJwsLX1AEAHGIahmpoaVVRUuCzl5eWqqKhQbW2tbDabyxIQENBsvaXn2nu9vX3ae60j+7R0vF153Pg5wzBkt9vV0NCg+vp6NTQ0tLl0ZJumy7XXXqv4+PiunuZOIYx4o4QEKTRUqqmRjh+XBg+2uiIAfqqhoUGVlZUuoaKtpaGhweqS0YqqqirLPpsw4o0CAszhvfv3myNqCCNAl9jtducPak1NjQIDAxUcHKygoCCXPwMDA1v8F3BPaWhoUG1trWpqalRTU9Puem1trcu/rh21Nv2zI+st7WMYhqqqqlzCRWd+wEJCQhQZGdlsCQsLk2EYzlaAttZbe93d7Tuyb0frafp3ozsf22w2BQYGtrkEBQU51wMCApo9195+/fr1c/tcdhfCiLdKTzfDCP1GgGbsdruqqqqcAaPx0vg5x7o7P6hBQUHNQkpHnnOsG4bRLEi0FjK8pRXBZrO1GC5aW4K5tIwmCCPeik6s+JZhGGpoaFBdXV2HF8MwFBISotDQUIWEhLgsjZ/r7ZaApsfVWv1NQ0bTgFFZWdni9ff2REREKCwsTPX19aqvr1ddXZ3q6+td3svxWnV1dXcebpsCAwMVGhrqPDeN15s+DggwxyU07mvg0NXnbDabwsPDFRUV5RIuwsPDLft7At9AGPFWhBGfVFdXpwMHDujo0aOqra3tcLjoKQEBAa0GlbZCjCNIOH7QGy8tPdfS0h2tAuHh4YqIiHD+aDZeb/pceHi484e8MUcTfONw0vTPjr5WX18vSc0CRNP1ps8FBgZ2+b8F4MkII97KEUa++koyDIl/lXit2tpaff3118rNzdX+/fu7FC4CAgIUHBzc7mKz2VwuDzjWHY8dP5p2u13V1dW92grQEsclDscSERHhDBGtBYyIiIhu+RFvfK0+NDS0G44GQFOEEW/1/e+bI2o+/1xatUq69VarK4Ibqqur9fXXX2vPnj06ePCg88dfkmJjYzVs2DBFRUV1KFg4FkcntO5gt9tdAkproaXpNo7nHaGoaYhoXKs7x8UlAMC32YzOXFjtZWVlZYqNjVVpaaliYmKsLsdzPPKIdO+9UkyMtGuXlJZmdUWWMQxD5eXlKi4uVlFRkUpKShQUFKTU1FSlpqaqX79+lv+gVVVVae/evcrNzdXBgwdlt9udr/Xt21cZGRkaOXKkUlJSLK8VALpDR3+/aRnxZosWSa+9Jn3yiXT77dI//+nzl2sMw1BFRYWKioqcwaO4uFjFxcVtXkoIDQ1VSkqKM5ykpqYqLi6ux3/0KyoqtHfvXu3Zs0eHDx92CSDx8fEaOXKkRo4cqcTERAIIAL9Fy4i327dPGjdOqq6Wnn1WmjfP6oq6hSN0NA0cxcXFrQ7DtNls6tu3rxISEpSQkKCamhrl5+crPz/f5TKIQ3h4uEs4SU1N7Za/X2fPnlVubq5yc3N15MgRlxEJSUlJzhaQhISELn8WAHiyjv5+E0Z8wWOPSXffLUVHSzt3nuvc6iVaCh1FRUVtzv3QOHQkJiYqISFB8fHxCgpq3thnt9tVXFysEydOKC8vT/n5+SooKHBppXCIiopqFlAiIyPbPYbS0lLl5uZqz549OnbsmMtrKSkpGjlypDIyMiydVAgAehthxJ80NEiTJ0ubN0uXXSZt2OCRl2sMw1BZWZmOHz+u48ePq6CgQEVFRaqsrGx1nz59+jjDhmOJj4/v8qRJ9fX1KioqUl5enk6cOKH8/HwVFRW1ODdFbGxss4ASFham06dPa8+ePcrNzVVeXp7LPgMGDFBGRoYyMjLUp0+fLtUKAN6KMOJv9u+Xxo6Vqqqkp56S7rjD6opUW1ur/Px8Z/g4fvy4ysvLW9w2Li7OJXQkJiZ2S+hwR11dnQoKCpytJ3l5eTp58mSL20ZFRTU7loEDBzpbQPh7CgCEEf/0P/8jLVwoRUaal2t68Z41hmHo1KlTLsGjsLCwWUuDzWZTcnKy+vfvr/79+ztDR0hISK/V6g5Hv5PGAeXMmTOSzGMZNGiQswUkKirK2mIBwMMQRvyR3S5NmSL961/SpZdK771n3lSvB1RXVysvL0/Hjx93/tlSH4+oqCilpaWpf//+GjBggFJTU73+vhSVlZUqLi5WfHx8h/qTAIC/Ioz4q4MHpTFjpMpK6cknpfnzu/yWjg6gjhaPvLw8FRcXN9suMDBQqampzuAxYMAAxcTEMGQVAPwUYcSfrVghLVggRUSY08Wfd16Hd62pqVFJSYlOnjyp4uJi5eXlKS8vT7W1tc227dOnjwYMGOAMH8nJydxDAwDgxKRn/uzOO6V//EP64ANzmvhNm1wu1zQ0NOj06dM6efKkTp48qZKSEp06dUolJSWqqKho8S1DQkKc/TwcrR5cogAAdAfCiC8KCJDxpz+pPCtLJ48dU8kf/qCTY8c6w8fp06fbvL16VFSU+vXrp379+ik1NVUDBgxQQkJCi3c0BQCgqwgjXq6mpsYZMpoutb/8pblRVZU5ZXwjwcHBzsDhWOLj49W3b1+FhYVZcCQAAH9FGPFSJSUlevXVV1VUVNTqNjabTXGVlYo/dkx9Q0MVf8cd6peQoH79+ik6OpqOpQAAj0AY8UJ1dXUuQSQyMrLFVo4+ffoo8PhxKTNTOntWGj3avLkeAAAehDDihdavX6+ioiJFRkZq3rx5iouLa33j9HTz3jW33y79+tfSlVdKI0b0Wq0AALSHHole5ssvv1ROTo5sNpuuv/76toOIw223SdOnm3f2vfVW8142AAB4CMKIFykuLta6deskSZMnT9bgjk73brNJzz4rxcSYHVkfe6wHqwQAwD2EES9RW1urV199VXV1dRoyZIi+973vufcGaWnS44+b60uWSHv2dHuNAAB0BmHES6xfv17FxcWKiorSdddd17k5P265xewzUlNjrtfXd3eZAAC4jTDiBXJycvTll186+4l0+u6wNpv0zDNSXJz0+efSI490a50AAHQGYcTDFRUVOfuJTJkyRYMGDeraG/bvL/3v/5rry5ZJO3d27f0AAOgiwogHc/QTqa+v13nnned+P5HW/PSn0jXXSHV15uWaurrueV8AADqBMOKhDMPQunXrVFJSoujoaF133XXdN2OqzSY9/bTUp4+0fbv0u991z/sCANAJhBEPtWPHDn311VfOfiLdfofclBTpySfN9d/+Vvryy+59fwAAOogw4oEKCwv1z3/+U5J06aWXKj09vWc+aM4c6brrzl2uqa3tmc8BAKANhBEPU1NT4+wncv755+u73/1uz32YzSY99ZTUr5+UkyP913/13GcBANAKwogHcfQTOXnyZPf3E2lNUpK0YoW5/p//Ke3Y0bOfBwBAE4QRD7J9+3bt3LlTNptNP/rRjxQREdE7H3zDDdKPfmROgnbzzVyuAQD0KsKIhygoKHD2E7nssss0cODA3vtwm0364x+lhARz3pGHHuq9zwYA+D3CiAdw9BNpaGjQ0KFDNWnSpN4vIiHB7D8iScuXS9u29X4NAAC/RBixmGEYevPNN3Xq1CnFxMQoOzu75/uJtOb666Uf/1hqaDAv19TUWFMHAMCvEEYs9sUXX2j37t0KCAjo3X4irXniCSkx0byr74MPWlsLAMAvEEYslJ+fr7fffluS2U8kLS3N4ookxcdLK1ea67//vfTyy5JhWFsTAMCnEUYsUl1d7ewnMmzYMGVlZVld0jnXXWfev8ZuNy/bzJgh7d9vdVUAAB9FGLGAo5/I6dOnFRsba20/kdY8+6y0ZIkUEiK98440erT5uLLS6soAAD6GMGKBzz//XHv27HH2EwkPD7e6pObCwsx71uzaJU2fbs498h//IY0cKa1dy6UbAEC3IYz0shMnTujdd9+VJE2bNk0DBgywuKJ2DB0q/fOf0muvSQMHSkeOSNnZ0tVXSwcPWl0dAMAHEEZ6UeN+IiNGjNAll1xidUkdY7OZ/Uj27JH+/d+l4GBp/Xpp1ChzxE1VldUVAgC8GGGklxiGoTfeeENnzpxRXFycrr32Ws/rJ9KeyEjz/jU7d0o/+IE5D8lvfmOGkrfesro6AICXIoz0ks8++0y5ubme3U+ko4YPNzu1vvqqNGCAdOiQdM010rXXmusAALiBMNIL8vLynP1ELr/8cvXv39/iirqBzWbeXC83V7rvPikoSHrzTbOD60MPSdXVVlcIAPAShJEeVlVVpb///e+y2+3KyMjQxRdfbHVJ3SsqSnr4Yemrr6SpU80QsnSpORT42xv/AQDQFsJIDzIMQ2vXrvXufiIdlZEhvfee9NJLUmqqOdLmyiulH/7QHIEDAEArCCM96NNPP9W+ffsUGBioWbNmKSwszOqSepbNJs2eLe3dK91zj3np5vXXzaDyX//FjfcAAC0ijPSQyspKvffee5LMfiKpqakWV9SLoqOlRx6RcnKkyZPNob+//rU0Zoz0bd8ZAAAcCCM9ZPfu3WpoaFBycrIuuugiq8uxxqhR0gcfSH/7m5ScLH39tTmb66xZ0rFjVlcHAPAQnQojK1as0KBBgxQWFqaJEyfqs88+a3XbVatWyWazuSw+f7lC0ldffSVJGjNmjO/2E+kIm036yU+kffukX/1KCgyU/v53acQIc3r548etrhAAYDG3w8jLL7+sRYsWadmyZdq+fbvGjh2r6dOnq6ioqNV9YmJilJ+f71yO+HiHxlOnTun48eOy2WzKzMy0uhzPEBMjPfaYtH279N3vmjfcW7JESkuTJk0yX/PxvxcAgJa5HUYee+wx3X777br11ls1cuRIrVy5UhEREXruueda3cdmsyk5Odm5JCUldaloT+doFRkyZIiioqIsrsbDjBkjffSRtHq1GUpsNmnrVunuu6VBg6SLL5Z+/3vuewMAfsStMFJbW6svvvhC06ZNO/cGAQGaNm2atm7d2up+5eXlSk9PV1pammbOnKndu3e3+Tk1NTUqKytzWbyFYRgul2jQAptNuvFG6V//Mi/TPPmkNGWKFBAgff65OYna+edLF1xgjsLZt8/qigEAPcitMFJSUqKGhoZmLRtJSUkqKChocZ/hw4frueee09q1a7V69WrZ7XZNmjRJx9voK7B8+XLFxsY6l7S0NHfKtFReXp5Onz6t4OBgjRgxwupyPF9qqjR/vtnR9cQJaeVKado0s2/Jjh3mKJwRI6TMTPM+OLt3S4ZhddUAgG7U46NpsrKyNHfuXI0bN06TJ0/Wa6+9poSEBD399NOt7rN48WKVlpY6l2NeNPLC0SqSkZGhkJAQi6vxMklJ0i9+IW3YIBUUSH/6kzRjhjlfya5d5h2CR482p5xfskT68kuCCQD4ALfCSHx8vAIDA1VYWOjyfGFhoZKTkzv0HsHBwRo/frwOHDjQ6jahoaGKiYlxWbxBQ0ODdu3aJUl0XO2q+Hjpttuk9euloiLpL38xb8YXEmJOqvYf/yGNGycNHSrdf7+0bRvBBAC8lFthJCQkRBMmTNDGjRudz9ntdm3cuFFZWVkdeo+Ghgbt3LlTKSkp7lXqBQ4cOKCqqipFRkZqyJAhVpfjO/r0kebOld54QyouNuctue46KSzM7Oj6u99JF10kDR5szvz6ySeS3W511QCADnL7Ms2iRYv07LPP6i9/+Ytyc3N15513qqKiQrfeeqskae7cuVq8eLFz+9/+9rd699139c0332j79u366U9/qiNHjmjevHnddxQeYufOnZLMVpGAAOaT6xExMea8Ja+9ZgaTV16RbrhBiogwhwY/+qiUlSWlp0vz5kkrVkhbtkjl5VZXDgBoRZC7O8yePVvFxcVaunSpCgoKNG7cOL399tvOTq1Hjx51+SE+ffq0br/9dhUUFKhPnz6aMGGCtmzZopEjR3bfUXiA6upq7ft21AejaHpJVJQ5m+usWea8Je+8Y06o9uab5iidP//53LY2mzRsmDR+vHl5Z/x4c0lIsKx8AIDJZhief6G9rKxMsbGxKi0t9dj+Izt27NAbb7yh+Ph4/fKXv/TvWVetVl1t3kH4k0/METk7dkj5+S1v27//uWDiCCmDBpnhBQDQJR39/Xa7ZQQtY/p3DxIWJl19tbk4FBaeCyY7dpg38du/X8rLM5e33jq3bVyca+vJuHHmnYeD+LoAQE/g/67doLS0VIcPH5bEKBqPlZQkXXGFuTicPWsOD24cUnbvls6ckTZtMheH0FBzrhNHQBk/3hxi7KEtdQDgTQgj3cDRcTU9PV1xcXHWFoOOi442p6T/7nfPPVdbK+3Z07wVpbzcHD68bZvre/Tvb7aaNF0SE7nUAwAdRBjposbTv9Mq4gNCQszLMuPGSd+OEJPdbg4hzslxDSgFBecu87z3nuv79OnTckhJTzenvQcAOBFGuqiwsFDFxcUKDAzUqFGjrC4HPSEgwJxcbehQc+SOw+nTUm5u8+XwYfO1LVvMpbHwcGn48OYhZehQMwgBgB8ijHSRo1Vk2LBhCgsLs7ga9Ko+faRJk8ylsaoq8+Z+TUPK/v3mazk55tJYYKB03nmuAWX4cHPh0h8AH0cY6QK73e6c/p25ReAUHn7uUk9j9fXSoUMtt6acPSt9/bW5rF3rul9CwrlgMny4OV/K8OHSkCG0pgDwCYSRLjh8+LDOnj2r8PBwDR061Opy4OmCgs5d7rn22nPPG4Z5x+KmAeXrr83ni4vN5eOPXd8vMNAMJI5w0jioJCfTgRaA1yCMdIHjEs3IkSMVGBhocTXwWjabOSqnf39p2jTX1xq3mOzbZy6O9YoK89LP/v3SunWu+8XEmMGkcVAZPtwMQpGRvXdsANABhJFOqqurU25uriQu0aAHRUdLEyaYS2OO1pTGIcURVA4dksrKWh6KLEkDBpj9U1JSzBYUx9L4cXw8o34A9BrCSCft3btXtbW1iouLU1pamtXlwN80bk259FLX12pqzKHILQWVkhLzvj3Hj7f9/oGB5kRxLQWVpo9paQHQRYSRTnJMdMb07/A4oaHm7LAt3Yzy1CkzmBw5Ys6TUlBg3ren8XpJidTQYLa8nDjR/udFRbUcVlJSpNTUc+v9+tGPBUCLCCOdUF5ergMHDkhiojN4mb59pawsc2lNXZ3ZYbZxSGkptOTnm0OVy8vP9V1pS0jIuaDSOKQ0XecSEeB3CCOdsHv3bhmGodTUVMXHx1tdDtC9goPNUJCa2vZ2hmEGkdaCimM5cUI6edKcav/oUXNpS1CQeYmotcCSkmJOt5+QYN4UEYDXI4x0QuM79AJ+y2YzO9hGR5ujdNpSU2PeOfnECdeQ0nS9uNicj8UxzX57oqPNUNJ4cQSVlpaIiO45dgDdijDippKSEp04cUI2m02jR4+2uhzAO4SGSgMHmktb6urM0NK0ZaW10HL2rLl8803H6oiM7Fh46dvX7OMSG8slI6AXEEbc5GgVOf/88xXJKAKgewUHm0OPBwxoezvDkEpLpaKic5PCOZaWnisuNi8TVVSYy+HDHavHZjOn/XeEk759XZfWnouNNUckAegQwogbDMNwGUUDwCI2m3nPnrg4c2K39hiGOfdKR4PLqVNmaDEMc/3UKenbTutu1ddSaOnTxwwrTZeYmHPrERGMPIJfIYy44dixYzpz5oxCQkI0fPhwq8sB0FE227kf+vPP79g+NTXm3ZdPnTI74DpCiWNp+pzjcXm5GWJOnzaXgwfdrzcoyDWcuLPExJh9aaKiaJ2B1yCMuMFxiSYjI0PBwcEWVwOgR4WGnps3xR21tWYIaSmsnDwpnTljXmJqvJSVnfvTbjf7wzj264rIyHOdjB0hpSOPm74WHc1NGdGjCCMdVF9fr927d0viEg2ANoSEmEOTk5Lc39cxXLppWHFnOXvWDDPSuT4yBQVdP67Q0I4Fl468FhbGZSi4IIx00IEDB1RdXa3o6GgNGjTI6nIA+KLGw6Xb68TbGsMwLzGVlZ0bbdR43d3H1dXm+9bUmEtJSdePMzCw5eASFWW25jj+bG1p7XVab7wWYaSDHJdoRo8erQCG+gHwVDab2fIQFmYOW+6qurrWQ0vT59oLOeXl5ns2NJiXq86c6Xp9jQUFdSzMRESYi2O96Z+tPUfY6TGEkQ6oqqrS119/LYlLNAD8THDwuZFAXWW3m4GkteBSXn7u0lLj9faWujrz/evrz12u6glBQW2HlvDwc0HQsd70T3deCw31m3luCCMdsGfPHjU0NCgxMVHJ7nZmAwCYAgLMyzExMeYdp7tLXV3HA0xl5bk/G6+39WdDg/k59fVmaCor677a2xMaei6cOFp9ml7OavpcR9bDwz2q3w5hpAOYWwQAPFhw8Ll5Z3pCbW3bgcWxXlVl9rFx/Nl4vaOvVVWZLUgOjr463d3aY7M1DynPPSddcEH3fk4HEUbacebMGR05ckQSd+gFAL8UEmIuPRV2mqqvbzmoOFp/HC1AjvWmj9tar6w0P8MxcsvRj0dyDUG9jDDSDkeryODBgxUTE2NxNQAAnxcUdG6UUXdraDjXktM0sHRkNuMeQhhpg2EYzlE0tIoAALyeY1h1TwSdLvCPbrqdlJ+fr5KSEgUFBSkjI8PqcgAA8EmEkTY4WkWGDx+usLAwi6sBAMA3EUZaYbfbtWvXLkmMogEAoCcRRlrxzTffqKKiQuHh4TrvvPOsLgcAAJ9FGGlF4+nfA7kNNwAAPYYw0oLa2lrt3btXEpdoAADoaYSRFuzdu1d1dXXq27ev+nfnlMUAAKAZwkgLGs8tYvOgufsBAPBFhJEmzp49q2+++UYSl2gAAOgNhJEmdu3aJcMwNGDAAPXtjltmAwCANhFGmnBcoqFVBACA3kEYaaSoqEgFBQUKCAjQqFGjrC4HAAC/QBhpxHGH3qFDhyoiIsLiagAA8A+EkW8ZhuEMI9yhFwCA3kMY+daRI0dUWlqq0NBQDRs2zOpyAADwG4SRbzk6ro4cOVLBwcEWVwMAgP8gjEiqr6/Xnj17JDGKBgCA3kYYkfT111+rpqZGMTExSk9Pt7ocAAD8CmFETP8OAICV/D6MVFZWav/+/ZK4RAMAgBX8Pozs3r1bdrtdycnJSkxMtLocAAD8jt+HEeYWAQDAWn4dRk6fPq1jx47JZrMRRgAAsIhfhxFHx9XBgwcrOjra4moAAPBPfhtGDMPgDr0AAHiAIKsLsNIPfvAD7d69WyNGjLC6FAAA/JbfhhGbzaYRI0YQRAAAsJjfXqYBAACegTACAAAsRRgBAACWIowAAABLEUYAAIClCCMAAMBShBEAAGApwggAALBUp8LIihUrNGjQIIWFhWnixIn67LPP2tz+1Vdf1YgRIxQWFqbMzEytX7++U8UCAADf43YYefnll7Vo0SItW7ZM27dv19ixYzV9+nQVFRW1uP2WLVs0Z84c3XbbbdqxY4eys7OVnZ2tXbt2dbl4AADg/WyGYRju7DBx4kRddNFFevLJJyVJdrtdaWlp+n//7//p/vvvb7b97NmzVVFRobfeesv53CWXXKJx48Zp5cqVHfrMsrIyxcbGqrS0VDExMe6UCwAALNLR32+3WkZqa2v1xRdfaNq0aefeICBA06ZN09atW1vcZ+vWrS7bS9L06dNb3R4AAPgXt26UV1JSooaGBiUlJbk8n5SUpL1797a4T0FBQYvbFxQUtPo5NTU1qqmpcT4uKytzp0wAAOBFPPKuvcuXL9dvfvObZs8TSgAA8B6O3+32eoS4FUbi4+MVGBiowsJCl+cLCwuVnJzc4j7JyclubS9Jixcv1qJFi5yP8/LyNHLkSKWlpblTLgAA8ABnz55VbGxsq6+7FUZCQkI0YcIEbdy4UdnZ2ZLMDqwbN27UggULWtwnKytLGzdu1MKFC53PbdiwQVlZWa1+TmhoqEJDQ52Po6KidOzYMUVHR8tms7lTcpvKysqUlpamY8eO+UXHWH86Xo7Vd/nT8XKsvstfjtcwDJ09e1apqaltbuf2ZZpFixbp5ptv1oUXXqiLL75Yjz/+uCoqKnTrrbdKkubOnav+/ftr+fLlkqS77rpLkydP1qOPPqqrrrpKL730krZt26Znnnmmw58ZEBCgAQMGuFtqh8XExPj0X4am/Ol4OVbf5U/Hy7H6Ln843rZaRBzcDiOzZ89WcXGxli5dqoKCAo0bN05vv/22s5Pq0aNHFRBwbpDOpEmT9MILL+iBBx7Qv//7v2vo0KFas2aNRo8e7e5HAwAAH9SpDqwLFixo9bLMpk2bmj03a9YszZo1qzMfBQAAfJxf35smNDRUy5Ytc+mf4sv86Xg5Vt/lT8fLsfoufzve9rg9AysAAEB38uuWEQAAYD3CCAAAsBRhBAAAWIowAgAALOXzYWTFihUaNGiQwsLCNHHiRH322Wdtbv/qq69qxIgRCgsLU2ZmptavX99LlXbN8uXLddFFFyk6OlqJiYnKzs7Wvn372txn1apVstlsLktYWFgvVdx5Dz74YLO6R4wY0eY+3npeBw0a1OxYbTab5s+f3+L23nZOP/roI11zzTVKTU2VzWbTmjVrXF43DENLly5VSkqKwsPDNW3aNO3fv7/d93X3e98b2jrWuro63XfffcrMzFRkZKRSU1M1d+5cnThxos337Mx3oTe0d15vueWWZnVfccUV7b6vJ55Xqf3jbek7bLPZ9Mgjj7T6np56bnuKT4eRl19+WYsWLdKyZcu0fft2jR07VtOnT1dRUVGL22/ZskVz5szRbbfdph07dig7O1vZ2dnatWtXL1fuvg8//FDz58/XJ598og0bNqiurk6XX365Kioq2twvJiZG+fn5zuXIkSO9VHHXjBo1yqXujz/+uNVtvfm8fv755y7HuWHDBklqc94ebzqnFRUVGjt2rFasWNHi67///e/1v//7v1q5cqU+/fRTRUZGavr06aqurm71Pd393veWto61srJS27dv15IlS7R9+3a99tpr2rdvn6699tp239ed70Jvae+8StIVV1zhUveLL77Y5nt66nmV2j/exseZn5+v5557TjabTddff32b7+uJ57bHGD7s4osvNubPn+983NDQYKSmphrLly9vcfsbbrjBuOqqq1yemzhxovGLX/yiR+vsCUVFRYYk48MPP2x1m+eff96IjY3tvaK6ybJly4yxY8d2eHtfOq933XWXcd555xl2u73F1731nBqGYUgyXn/9dedju91uJCcnG4888ojzuTNnzhihoaHGiy++2Or7uPu9t0LTY23JZ599Zkgyjhw50uo27n4XrNDSsd58883GzJkz3XofbzivhtGxcztz5kxj6tSpbW7jDee2O/lsy0htba2++OILTZs2zflcQECApk2bpq1bt7a4z9atW122l6Tp06e3ur0nKy0tlST17du3ze3Ky8uVnp6utLQ0zZw5U7t37+6N8rps//79Sk1N1ZAhQ3TjjTfq6NGjrW7rK+e1trZWq1ev1s9+9rM2bxjpree0qUOHDqmgoMDl3MXGxmrixImtnrvOfO89VWlpqWw2m+Li4trczp3vgifZtGmTEhMTNXz4cN155506efJkq9v60nktLCzUunXrdNttt7W7rbee287w2TBSUlKihoYG5z1zHJKSklRQUNDiPgUFBW5t76nsdrsWLlyo73znO23eA2j48OF67rnntHbtWq1evVp2u12TJk3S8ePHe7Fa902cOFGrVq3S22+/raeeekqHDh3S9773PZ09e7bF7X3lvK5Zs0ZnzpzRLbfc0uo23npOW+I4P+6cu8587z1RdXW17rvvPs2ZM6fNm6i5+13wFFdccYX++te/auPGjfrd736nDz/8UDNmzFBDQ0OL2/vKeZWkv/zlL4qOjtYPf/jDNrfz1nPbWZ26Nw082/z587Vr1652ry9mZWUpKyvL+XjSpEnKyMjQ008/rYceeqiny+y0GTNmONfHjBmjiRMnKj09Xa+88kqH/rXhrf785z9rxowZbd6K21vPKc6pq6vTDTfcIMMw9NRTT7W5rbd+F3784x871zMzMzVmzBidd9552rRpky677DILK+t5zz33nG688cZ2O5Z767ntLJ9tGYmPj1dgYKAKCwtdni8sLFRycnKL+yQnJ7u1vSdasGCB3nrrLX3wwQcaMGCAW/sGBwdr/PjxOnDgQA9V1zPi4uI0bNiwVuv2hfN65MgRvffee5o3b55b+3nrOZXkPD/unLvOfO89iSOIHDlyRBs2bHD71vLtfRc81ZAhQxQfH99q3d5+Xh3+9a9/ad++fW5/jyXvPbcd5bNhJCQkRBMmTNDGjRudz9ntdm3cuNHlX46NZWVluWwvSRs2bGh1e09iGIYWLFig119/Xe+//74GDx7s9ns0NDRo586dSklJ6YEKe055ebkOHjzYat3efF4dnn/+eSUmJuqqq65yaz9vPaeSNHjwYCUnJ7ucu7KyMn366aetnrvOfO89hSOI7N+/X++995769evn9nu0913wVMePH9fJkydbrdubz2tjf/7znzVhwgSNHTvW7X299dx2mNU9aHvSSy+9ZISGhhqrVq0y9uzZY/z85z834uLijIKCAsMwDOOmm24y7r//fuf2mzdvNoKCgow//OEPRm5urrFs2TIjODjY2Llzp1WH0GF33nmnERsba2zatMnIz893LpWVlc5tmh7vb37zG+Odd94xDh48aHzxxRfGj3/8YyMsLMzYvXu3FYfQYXfffbexadMm49ChQ8bmzZuNadOmGfHx8UZRUZFhGL51Xg3DHDUwcOBA47777mv2mref07Nnzxo7duwwduzYYUgyHnvsMWPHjh3OESQPP/ywERcXZ6xdu9b46quvjJkzZxqDBw82qqqqnO8xdepU44knnnA+bu97b5W2jrW2tta49tprjQEDBhg5OTku3+GamhrnezQ91va+C1Zp61jPnj1r3HPPPcbWrVuNQ4cOGe+9955xwQUXGEOHDjWqq6ud7+Et59Uw2v97bBiGUVpaakRERBhPPfVUi+/hLee2p/h0GDEMw3jiiSeMgQMHGiEhIcbFF19sfPLJJ87XJk+ebNx8880u27/yyivGsGHDjJCQEGPUqFHGunXrernizpHU4vL88887t2l6vAsXLnT+t0lKSjKuvPJKY/v27b1fvJtmz55tpKSkGCEhIUb//v2N2bNnGwcOHHC+7kvn1TAM45133jEkGfv27Wv2mref0w8++KDFv7eOY7Lb7caSJUuMpKQkIzQ01Ljsssua/XdIT083li1b5vJcW997q7R1rIcOHWr1O/zBBx8436Ppsbb3XbBKW8daWVlpXH755UZCQoIRHBxspKenG7fffnuzUOEt59Uw2v97bBiG8fTTTxvh4eHGmTNnWnwPbzm3PcVmGIbRo00vAAAAbfDZPiMAAMA7EEYAAIClCCMAAMBShBEAAGApwggAALAUYQQAAFiKMAIAACxFGAEAAJYijAAAAEsRRgAAgKUIIwAAwFKEEQAAYKn/D8W0t82NJl+eAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from torchvision import transforms\n",
    "from Module import Utils as myutils\n",
    "\n",
    "# 定义模型\n",
    "class LeNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(LeNet, self).__init__()\n",
    "        self.structure = nn.Sequential(\n",
    "            nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5, stride=1, padding=2),\n",
    "            nn.Sigmoid(),\n",
    "            nn.AvgPool2d(kernel_size=2, stride=2),\n",
    "            nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0),\n",
    "            nn.Sigmoid(),\n",
    "            nn.AvgPool2d(kernel_size=2, stride=2),\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(in_features=16*5*5, out_features=120),\n",
    "            nn.Sigmoid(),\n",
    "            nn.Linear(in_features=120, out_features=84),\n",
    "            nn.Sigmoid(),\n",
    "            nn.Linear(in_features=84, out_features=10)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.structure(x)\n",
    "    \n",
    "# 图片预处理\n",
    "trans = transforms.Compose(\n",
    "    [\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Resize(size=(28,28))\n",
    "    ]\n",
    ")\n",
    "\n",
    "# 设置超参数\n",
    "batch_size = 128\n",
    "epochs = 20\n",
    "lr = 1.2\n",
    "\n",
    "# 获取数据集\n",
    "_, train_iter, test_iter = myutils.load_data_FMNIST('../data', batch_size, trans)\n",
    "\n",
    "# 实例化网络\n",
    "net = LeNet()\n",
    "\n",
    "# 创建优化器\n",
    "optimizer = torch.optim.SGD(params=net.parameters(),lr=lr)\n",
    "\n",
    "# 定义损失函数\n",
    "loss = nn.CrossEntropyLoss()\n",
    "\n",
    "# 开始训练\n",
    "myutils.train(net, train_iter, test_iter, epochs, optimizer, loss, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`conv1`：5*5 卷积，padding为2，stride为1，即原图中每个像素点都作为一次卷积中的中心像素点，故输出尺寸不变\n",
    "\n",
    "`nn.Conv2D`：只考虑单个图片的输入通道数和输出通道数，batch_size在创建该对象时无需考虑，在数据输入网络时自动计算，输入该层的形状为`[N, C, H, W]`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由LeNet网络结构可以发现：\n",
    "1. 卷积神经网络学习的是卷积层的模式（卷积核），越多的模式意味着可以匹配越多的特征，所以一般在卷积层对一张图像使用多通道的卷积核（以学习较多的模式），同时缩小图像的尺寸\n",
    "2. 在最后将多种模式用多个全连接层降维，逐步缩小到需要输出的尺寸"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "torch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
